Systems, methods, and media for outputting a dataset based upon anomaly detection

ABSTRACT

Systems, methods, and media for outputting a dataset based upon anomaly detection are provided. In some embodiments, methods for outputting a dataset based upon anomaly detection: receive a training dataset having a plurality of n-grams, which plurality includes a first plurality of distinct training n-grams each being a first size; compute a first plurality of appearance frequencies, each for a corresponding one of the first plurality of distinct training n-grams; receive an input dataset including first input n-grams each being the first size; define a first window in the input dataset; identify as being first matching n-grams the first input n-grams in the first window that correspond to the first plurality of distinct training n-grams; compute a first anomaly detection score for the input dataset using the first matching n-grams and the first plurality of appearance frequencies; and output the input dataset based on the first anomaly detection score.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.13/769,774, filed Feb. 18, 2013, which is a continuation of U.S. patentapplication Ser. No. 12/280,969, filed Nov. 24, 2008 and issued Feb. 19,2013 as U.S. Pat. No. 8,381,299, which is the U.S. National Phaseapplication Under 35 U.S.C. § 371 of International Application No.PCT/US2007/005408, filed Feb. 28, 2007, which claims the benefit of U.S.Provisional Patent Application No. 60/778,008, filed Feb. 28, 2006, andU.S. Provisional Patent Application No. 60/790,626, filed Apr. 10, 2006,each of which is hereby incorporated by reference herein in itsentirety.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

This invention was made with government support under Grant No. AROW911NF-04-1-0442 awarded by the Army Research Office (ARO). Thegovernment has certain rights in the invention.

TECHNICAL FIELD

The disclosed subject matter relates to systems, methods, and media foroutputting a dataset based upon anomaly detection.

BACKGROUND

Network data traffic models can be used to show unique characteristicsof specific network data traffic. These models can also be used fordetecting network data traffic content anomalies, such as maliciouscode. This is because the characteristics of normal data traffic differfrom the characteristics of data traffic harboring malicious code, suchas viruses, worms, Trojan horses, spyware, and/or other data that cancause harmful effects. Some network content anomaly detectors thatgenerate, train, and/or use such models can monitor and detect maliciouscode hidden in network traffic. However, they can still be vulnerable tomimicry attacks and training attacks.

A mimicry attack can be defined as a willful attempt to craft maliciouscode, or a package in which it is embedded, to look normal with respectto a network data traffic model associated with an anomaly detector,thereby managing to escape the detector's scrutiny. For example, anattacker can craft malicious code by padding the malicious code with alarge amount of data bytes that can compose a normal distribution. Forinstance, one network worm designed to monitor a flow of byte streamsinto a network can craft an attack worm to appear normal to the detectorthat monitors the network. Because the network worm can monitor andlearn the characteristics of the normal data flow into the network thatmay also be being used by the detector to train its data traffic models,the characteristics of the attack worm's composition may appear to benormal to the detector when the detector compares the characteristics ofthe attack worm to the characteristics of the normal network datatraffic.

A training attack is an attack whereby an attacker sends a stream ofdata that deviates from the normal data incrementally or continuously toa target site in order to influence the site's content anomaly detectorsuch that the detector would regard the attacker's malicious code asnormal. A content anomaly detector can, and often does, use incomingdata traffic to train its content anomaly detection model uponconfirming that the data traffic does not cause harmful effects. Anattacker can send a stream of data containing sparsely scattered bitsand pieces of malicious code that does not cause immediate harms to atarget site, but instead continuously trains that site's detector over aperiod of time to ignore the attack's threat.

SUMMARY

Systems, methods, and media for outputting a dataset based upon anomalydetection are provided. In some embodiments, methods for outputting adataset based upon anomaly detection are provided. These methods:receive a training dataset having a plurality of n-grams, whichplurality includes a first plurality of distinct training n-grams eachbeing a first size; compute a first plurality of appearance frequencies,each for a corresponding one of the first plurality of distinct trainingn-grams; receive an input dataset including first input n-grams eachbeing the first size; define a first window in the input dataset;identify as being first matching n-grams the first input n-grams in thefirst window that correspond to the first plurality of distinct trainingn-grams; compute a first anomaly detection score for the input datasetusing the first matching n-grams and the first plurality of appearancefrequencies; and output the input dataset based on the first anomalydetection score.

In some embodiments, methods for outputting a dataset based upon anomalydetection are provided. These methods: receive a training dataset havinga plurality of n-grams, which plurality includes a first plurality ofdistinct training n-grams each being a first size; select the firstplurality of distinct training n-grams on a random basis, pseudo-randombasis, or secret basis; receive an input dataset including first inputn-grams each being the first size; determine a first matching count ofthe first input n-grams that correspond to the first plurality ofdistinct training n-grams; determine a first total count of the firstinput n-grams; determine a first anomaly detection score using the firstmatching count and the first total count; and output the input datasetbased on the first anomaly detection score.

In some embodiments, methods for outputting a dataset based upon anomalydetection are provided. These methods: receive a first training datasethaving a plurality of n-grams, which plurality includes a firstplurality of distinct training n-grams each being a first size; receivea second training dataset having a plurality of n-grams, which pluralityincludes a second plurality of distinct training n-grams each being thefirst size; compute a first plurality of appearance frequencies, eachfor a corresponding one of the first plurality of distinct trainingn-grams; compute a first plurality of uniformities of distribution, eachfor a corresponding one of the first plurality of distinct trainingn-grams; compute a second plurality of uniformities of distribution,each for a corresponding one of the second plurality of distincttraining n-grams; determine a first plurality of most-heavily weightedn-grams from the first plurality of distinct training n-grams using atleast one of: the first plurality of appearance frequencies; the firstplurality of uniformities of distribution; and the second plurality ofuniformities of distribution; select a subset of the first plurality ofmost-heavily weighted n-grams, wherein the subset includes m n-grams andat least one of the n-grams in the subset is outside of the top m of thefirst plurality of most-heavily weighted n-grams; receive an inputdataset including first input n-grams each being the first size; obtaina subset of a second plurality of most-heavily weighted n-grams from thefirst input n-grams that correspond to the subset of the first pluralityof distinct training n-grams; classify the input dataset as containingan anomaly using the subset of the first plurality of most-heavilyweighted n-grams and the subset of the second plurality of most-heavilyweighted n-grams; and output a dataset based upon the classifying of theinput dataset.

In some embodiments, methods for constructing an anomaly detection modelare provided. Theses methods: receive a training dataset having aplurality of n-grams, which plurality includes a first plurality ofdistinct training n-grams each being a first size; compute a firstplurality of appearance frequencies, each for a corresponding one of thefirst plurality of distinct training n-grams; obtain a first pseudocount associated with the first plurality of appearance frequencies;compute a first total count of the plurality of n-grams that are eachthe first size; compute a first maximum possible count of distinctn-grams of the first size in the plurality of n-grams; compute a secondtotal count of the first plurality of distinct training n-grams; computea first smoothing factor; compute a first probability that the firstplurality of distinct training n-grams are found in the training datasetusing at least one of: the first plurality of appearance frequencies,the first pseudo count, the first total count, the second total count,and the first smoothing factor; compute a first consistency score of theplurality of n-grams that are each the first size using the firstmaximum possible count and the first probability; receive an inputdataset including first input n-grams each being the first size; obtaina second consistency score of the first input n-grams; classify theinput dataset using the first consistency score and the secondconsistency score; and output a dataset based upon the classifying ofthe input dataset.

In some embodiments, computer-readable media containingcomputer-executable instructions that, when executed by a processor,cause the processor to perform a method for outputting a dataset basedupon anomaly detection, are provided. These methods: receive a trainingdataset having a plurality of n-grams, which plurality includes a firstplurality of distinct training n-grams each being a first size; computea first plurality of appearance frequencies, each for a correspondingone of the first plurality of distinct training n-grams; receive aninput dataset including first input n-grams each being the first size;define a first window in the input dataset; identify as being firstmatching n-grams the first input n-grams in the first window thatcorrespond to the first plurality of distinct training n-grams; computea first anomaly detection score for the input dataset using the firstmatching n-grams and the first plurality of appearance frequencies; andoutput the input dataset based on the first anomaly detection score.

In some embodiments, computer-readable media containingcomputer-executable instructions that, when executed by a processor,cause the processor to perform a method for outputting a dataset basedupon anomaly detection, are provided. These methods: receive a trainingdataset having a plurality of n-grams, which plurality includes a firstplurality of distinct training n-grams each being a first size; selectthe first plurality of distinct training n-grams on a random basis,pseudo-random basis, or secret basis; receive an input dataset includingfirst input n-grams each being the first size; determine a firstmatching count of the first input n-grams that correspond to the firstplurality of distinct training n-grams; determine a first total count ofthe first input n-grams; determine a first anomaly detection score usingthe first matching count and the first total count; and output the inputdataset based on the first anomaly detection score.

In some embodiments, computer-readable media containingcomputer-executable instructions that, when executed by a processor,cause the processor to perform a method for outputting a dataset basedupon anomaly detection, are provided. These methods: receive a firsttraining dataset having a plurality of n-grams, which plurality includesa first plurality of distinct training n-grams each being a first size;receive a second training dataset having a plurality of n-grams, whichplurality includes a second plurality of distinct training n-grams eachbeing the first size; compute a first plurality of appearancefrequencies, each for a corresponding one of the first plurality ofdistinct training n-grams; compute a first plurality of uniformities ofdistribution, each for a corresponding one of the first plurality ofdistinct training n-grams; compute a second plurality of uniformities ofdistribution, each for a corresponding one of the second plurality ofdistinct training n-grams; determine a first plurality of most-heavilyweighted n-grams from the first plurality of distinct training n-gramsusing at least one of: the first plurality of appearance frequencies;the first plurality of uniformities of distribution; and the secondplurality of uniformities of distribution; select a subset of the firstplurality of most-heavily weighted n-grams, wherein the subset includesm n-grams and at least one of the n-grams in the subset is outside ofthe top m of the first plurality of most-heavily weighted n-grams;receive an input dataset including first input n-grams each being thefirst size; obtain a subset of a second plurality of most-heavilyweighted n-grams from the first input n-grams that correspond to thesubset of the first plurality of distinct training n-grams; classify theinput dataset as containing an anomaly using the subset of the firstplurality of most-heavily weighted n-grams and the subset of the secondplurality of most-heavily weighted n-grams; and output a dataset basedupon the classifying of the input dataset.

In some embodiments, computer-readable media containingcomputer-executable instructions that, when executed by a processor,cause the processor to perform a method for outputting a dataset basedupon anomaly detection, are provided. These methods: receive a trainingdataset having a plurality of n-grams, which plurality includes a firstplurality of distinct training n-grams each being a first size; computea first plurality of appearance frequencies, each for a correspondingone of the first plurality of distinct training n-grams; obtain a firstpseudo count associated with the first plurality of appearancefrequencies; compute a first total count of the plurality of n-gramsthat are each the first size; compute a first maximum possible count ofdistinct n-grams of the first size in the plurality of n-grams; computea second total count of the first plurality of distinct trainingn-grams; compute a first smoothing factor; compute a first probabilitythat the first plurality of distinct training n-grams are found in thetraining dataset using at least one of: the first plurality ofappearance frequencies, the first pseudo count, the first total count,the second total count, and the first smoothing factor; compute a firstconsistency score of the plurality of n-grams that are each the firstsize using the first maximum possible count and the first probability;receive an input dataset including first input n-grams each being thefirst size; obtain a second consistency score of the first inputn-grams; classify the input dataset using the first consistency scoreand the second consistency score; and output a dataset based upon theclassifying of the input dataset.

In some embodiments, systems for outputting a dataset based upon anomalydetection are provided. These systems including a digital processingdevice that: receives a training dataset having a plurality of n-grams,which plurality includes a first plurality of distinct training n-gramseach being a first size; computes a first plurality of appearancefrequencies, each for a corresponding one of the first plurality ofdistinct training n-grams; receives an input dataset including firstinput n-grams each being the first size; defines a first window in theinput dataset; identifies as being first matching n-grams the firstinput n-grams in the first window that correspond to the first pluralityof distinct training n-grams; computes a first anomaly detection scorefor the input dataset using the first matching n-grams and the firstplurality of appearance frequencies; and outputs the input dataset basedon the first anomaly detection score.

In some embodiments, systems for outputting a dataset based upon anomalydetection are provided. These systems including a digital processingdevice that: receives a training dataset having a plurality of n-grams,which plurality includes a first plurality of distinct training n-gramseach being a first size; selects the first plurality of distincttraining n-grams on a random basis, pseudo-random basis, or secretbasis; receives an input dataset including first input n-grams eachbeing the first size; determines a first matching count of the firstinput n-grams that correspond to the first plurality of distincttraining n-grams; determines a first total count of the first inputn-grams; determines a first anomaly detection score using the firstmatching count and the first total count; and outputs the input datasetbased on the first anomaly detection score.

In some embodiments, systems for outputting a dataset based upon anomalydetection are provided. These systems including a digital processingdevice that: receives a first training dataset having a plurality ofn-grams, which plurality includes a first plurality of distinct trainingn-grams each being a first size; receives a second training datasethaving a plurality of n-grams, which plurality includes a secondplurality of distinct training n-grams each being the first size;compute a first plurality of appearance frequencies, each for acorresponding one of the first plurality of distinct training n-grams;computes a first plurality of uniformities of distribution, each for acorresponding one of the first plurality of distinct training n-grams;computes a second plurality of uniformities of distribution, each for acorresponding one of the second plurality of distinct training n-grams;determines a first plurality of most-heavily weighted n-grams from thefirst plurality of distinct training n-grams using at least one of: thefirst plurality of appearance frequencies; the first plurality ofuniformities of distribution; and the second plurality of uniformitiesof distribution; selects a subset of the first plurality of most-heavilyweighted n-grams, wherein the subset includes m n-grams and at least oneof the n-grams in the subset is outside of the top m of the firstplurality of most-heavily weighted n-grams; receives an input datasetincluding first input n-grams each being the first size; obtains asubset of a second plurality of most-heavily weighted n-grams from thefirst input n-grams that correspond to the subset of the first pluralityof distinct training n-grams; classifies the input dataset as containingan anomaly using the subset of the first plurality of most-heavilyweighted n-grams and the subset of the second plurality of most-heavilyweighted n-grams; and outputs a dataset based upon the classifying ofthe input dataset.

In some embodiments, systems for constructing an anomaly detection modelare provided. These systems including a digital processing device that:receives a training dataset having a plurality of n-grams, whichplurality includes a first plurality of distinct training n-grams eachbeing a first size; computes a first plurality of appearancefrequencies, each for a corresponding one of the first plurality ofdistinct training n-grams; obtains a first pseudo count associated withthe first plurality of appearance frequencies; computes a first totalcount of the plurality of n-grams that are each the first size; computesa first maximum possible count of distinct n-grams of the first size inthe plurality of n-grams; computes a second total count of the firstplurality of distinct training n-grams; computes a first smoothingfactor; computes a first probability that the first plurality ofdistinct training n-grams are found in the training dataset using atleast one of: the first plurality of appearance frequencies, the firstpseudo count, the first total count, the second total count, and thefirst smoothing factor; computes a first consistency score of theplurality of n-grams that are each the first size using the firstmaximum possible count and the first probability; receives an inputdataset including first input n-grams each being the first size; obtaina second consistency score of the first input n-grams; classifies theinput dataset using the first consistency score and the secondconsistency score; and outputs a dataset based upon the classifying ofthe input dataset.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a simple illustration of a method for detecting contentanomalies in accordance with some embodiments of the disclosed subjectmatter.

FIG. 2 is a simple illustration of a method for generating, training,and applying a frequency distribution-based anomaly detection model inaccordance with some embodiments of the disclosed subject matter.

FIG. 3 is a simple illustration of a method for generating, training,and applying a binary-based anomaly detection model in accordance withsome embodiments of the disclosed subject matter.

FIG. 4 is a simple illustration of a method for training a frequencydistribution-based anomaly detection model in accordance with someembodiments of the disclosed subject matter.

FIG. 5 is a simple illustration of a method for classifying an inputdataset using a trained anomaly detection model in accordance with someembodiments of the disclosed subject matter.

FIG. 6 is a simple illustration of a method for training a binary-basedprobabilistic anomaly detection model in accordance with someembodiments of the disclosed subject matter.

FIG. 7 is a schematic diagram of a system for outputting a dataset basedupon anomaly detection in accordance with some embodiments of thedisclosed subject matter.

DETAILED DESCRIPTION

Systems, methods, and media for outputting a dataset based upon anomalydetection are provided. In some embodiments of the disclosed subjectmatter, systems, methods, and media are provided for generating and/ortraining frequency distribution-based content anomaly detection modelsthat use a mixture of higher order n-grams.

Frequency distribution-based content anomaly detection models are modelsthat represent the frequency distribution of the distinct n-gramsgenerated from training datasets.

An n-gram is a sequence of a number of consecutive tokens in a stream oftokens, such as one or more bytes, strings, characters, or othersuitable symbols and units of different types or definitions. Forexample, a 2-gram in bytes is a sequence of two bytes in a stream ofbytes. A higher order n-gram is an n-gram of size greater than a firstorder n-gram (i.e., 1-gram), such as a fifth order n-gram (i.e.,5-gram). A mixture of higher order n-grams is a group of higher ordern-gram of different sizes, such as a group of 3-grams, 5-grams, and9-grams.

Using a mixture of higher order n-grams in frequency distribution-basedcontent anomaly models is beneficial because it can capture thecontinuity between continuous bytes and contain more content informationthan a first order n-gram model, or even a fixed higher order n-grammodel.

In some embodiments of the disclosed subject matter, systems, methods,and media are provided for generating and/or training network datatraffic models using randomization processes. The randomizationprocesses can help thwart malicious efforts in crafting mimicry attacksbecause the information that is crucial to attackers is kept secret. Forexample, when one or more randomly selected parts of the payload of anetwork data packet are used for modeling the packet instead of usingthe whole packet payload, it is harder for an attacker to craft amalicious worm. For instance, in such cases, the attacker does not knowwhere and how to pad the worm's content to evade detection even if theattacker can monitor and learn the full knowledge of the content flow.

In some embodiments of the disclosed subject matter, systems, methods,and media are provided for generating and/or training binary-basedcontent anomaly detection models. A binary-based content anomalydetection model is based on the presence and/or absence of each ofdistinct n-grams in a training dataset. The model can be referencedduring a detecting or testing phase to detect the payload of a datapacket containing one or more never-before-seen n-grams, which arelikely to contain malicious code.

In some embodiments of the disclosed subject matter, systems, methods,and media are provided for computing a content anomaly detection score.A content anomaly detection score can be used to measure the likelihoodof a dataset containing malicious code. It can also measure thelikelihood of a dataset containing no malicious code. The score can becomputed, for example, by counting the occurrences of never-before-seenn-grams in a dataset and dividing the count by the total number ofn-grams in the dataset. Based on the score (for instance, if the scorewere high), a decision to examine the dataset further, rather thanprocessing requests in the dataset, can be made.

FIG. 1 is a simple illustration of a method 100 for detecting contentanomalies. As shown at 102, a content anomaly detection model isgenerated and trained. In some embodiments, a frequencydistribution-based detection model can be generated and trained, forexample, as described below in connection with part 201 of FIG. 2 andFIG. 4. In some embodiments, a binary-based detection model can begenerated and trained, for example, as described below in connectionwith part 301 of FIG. 3 and FIG. 6.

At 104, the content anomaly detection model is used to detect contentanomalies. In some embodiment, a frequency distribution-based anomalydetection model is used to detect content anomalies, for example, asdescribed below in connection with part 203 of FIG. 2 and FIG. 5. Insome embodiments, a binary-based anomaly detection model is used todetect content anomalies, for example, as described in connection withpart 303 of FIG. 3.

FIG. 2 is a simple illustration of a method 200 for generating,training, and applying a frequency distribution-based anomaly detectionmodel in accordance with some embodiments. At 202, a training dataset isreceived. In some embodiments, the training dataset can include one ormore network data packets or data frames. In some embodiments, thetraining dataset can include one or more files that contain varioustypes of data, such as text, graphic images, sound samples, videosamples, computer-executable codes, other suitable types of data, and/orone or more combinations thereof. In some embodiments, the trainingdataset can also include a stream of data in bytes or a stream of tokensin one or more communication sessions.

In some embodiments, before it is used for training anomaly detectionmodels, the training dataset can be checked to ascertain its validity toward off a potential training attack. Such a validation effort can alsohelp avoid inadvertently immunizing one or more instances of maliciouscode from being detected. For example, a training dataset can beprocessed to determine whether it harbors any data that produceserroneous or otherwise unusual or invalid outcomes. Suppose, forinstance, that the training dataset includes a Portable Document Format(PDF) file. The PDF file can be opened using an instance of an Adobe®Acrobat Reader running in a sandbox while the sandbox is beingmonitored. By opening the file in the sandbox while monitoring the stateof the sandbox, for instance, the server can determine whether the PDFfile contains malicious code.

At 204, an appearance frequency of each of distinct n-grams contained inthe training dataset is computed. In some embodiments, for example, eachof the distinct n-grams is saved in a data structure, such as a Bloomfilter or a hash table, when it is observed for the first time and acounter associated with that n-gram is incremented for each observationof the n-gram.

A Bloom filter can be defined as a bit array of m bits, wherein anindividual bit j is set to 1 if the hash of an input value (e.g., inputvalue mod n) is j. It acts as a one-way data structure that can containmany items within a storage capacity that is orders-of-magnitude smallerthan the maximum dimension of the stored items.

As the order of n-grams increases, the memory capacity necessary tohandle higher order n-grams increases exponentially. For example, fifthorder grams (i.e., 5-grams) of bytes can take 256⁵, or 1024 billion,different combinations of byte values.

A Bloom filter may contain false positives if a collision occurs whileperforming a check operation. Collisions may occur because two distinctinputs into a hash function may produce identical outputs. Suppose, forexample, there are two distinct n-grams, A (which occurs in the trainingdataset only) and B (which occurs in the input dataset only), which bothproduce the same hash value. Because A is in the training dataset, theBloom filter contains a bit set for A. If the Bloom filter is checkedfor B, however, because B hashes to the same value, then B can bemistakenly believed to be represented in the Bloom filter. This is afalse positive.

In some embodiments, a grouped n-gram can be used to alleviate suchmemory capacity requirements by treating one group of n-grams as onen-gram by defining the one n-gram as not considering the order of thecharacters therein. For example, a group of 3-grams, such as abc, acb,bac, bca, cab, and cba, can be grouped into one representative 3-gram,such as abc. As can be seen, a grouped n-gram can save memory usage. Forexample, the size of 5-grouped-grams in bytes is 120 times smaller than1024 billion.

Grouping n-grams can also help reduce potential false positivedetections. For example, in HTTP requests, often there are long “Range”attributes, such as:

-   -   Range: bytes=2056143-2057186, 2056141-2056142, 2057187-2059870,        2080621-2082690, . . . .        Such requests can be detected because, even though the exact        digit sequences being checked may never appear in a training        dataset, sequences in the same range as the digit sequences do        appear during the training phase. Thus, using grouped n-grams        can reduce false positive detections because the never-appearing        digit sequence will not be considered to be anomalous.

At 206, an input dataset is received. Like the training dataset, theinput dataset can be network data packets or frames, files containingdata in variety of types and formats, or a stream of bytes or tokens ofdifferent lengths. In some embodiments, the input dataset is a testdataset that is designed to measure how well a content anomaly detectionmodel is trained. In some embodiments, the input dataset is a stream ofincoming bytes that should be scrutinized before reaching itsdestination.

At 208, a window is applied to the input dataset and a set of distinctn-grams within the window is identified. Some types of malicious attackshide malicious code inside a data packet having large amount of normaldata. Using a window can help capture such malicious code. In someembodiments, the window is a sliding window.

At 210, an anomaly detection score of the input dataset in the window iscomputed using the set of distinct n-grams and the appearance frequencyof each distinct n-gram in the set. In some embodiments, the anomalydetection score is computed using the following formula:

${{{Detection}\mspace{14mu}{Score}} = \frac{\sum\limits_{i}{f\left( g_{i} \right)}}{T_{ngrams}}},$where g_(i) is a distinct n-gram contained in an input dataset, f(g_(i))is an appearance frequency of the distinct n-gram, g_(i), computedduring the training phase, and T_(n-grams) is the total number ofn-grams in the input dataset.

At 212, the anomaly detection score is stored. In some embodiments, adata structure, such as an array, a list, or any other suitable datastructure that can hold multiple elements, is used to store the score.For example, the detection score can be stored in a list.

At 214, it is determined whether the window has reached the end of theinput dataset. In some embodiments, the size of the window is less thanthe size of the input dataset and, therefore, the end of the inputdataset can be reached after sliding the window ahead, at 216, a numberof times.

If it has been determined at 214 that the window has not reached the endof the input dataset, the window is slid ahead at 216 and 208, 210, 212,214, and 216 or 218 are repeated. In some embodiments, the window isslid ahead by one token space or data unit. For example, the window canbe slid ahead by one byte. In some embodiments, the window is slid aheadby multiple token spaces or data units.

If it is determined at 214 that the window has reached the end of theinput dataset, however, the highest detection score can be selected at218 from the computed anomaly detection scores. A higher score may beselected because a higher score may indicate that the input dataset isnormal. This is because high scores indicate that more n-grams that havebeen observed during the training phase are also in the input dataset.

In some embodiments, the anomaly detection scores can be sorted using asorting algorithm. In some embodiments, for example, the highest scorecan be kept at the beginning of a list. Whenever a new anomaly detectionscore is computed, it can be compared with the highest score thus far.If the new score is greater, the new score is added at the beginning ofthe list. Otherwise, the new score is added at the end of the list. Inthis way, the highest detection score can be easily located.

FIG. 3 is a simple illustration of another method 300 for generating,training, and applying a binary-based anomaly detection model inaccordance with some embodiments. As shown, at 302, a training datasetis received. As in 202, the training dataset can be network data packetsor frames, files containing data in variety of types and formats, or astream of bytes or tokens of different lengths. Also, as in 202, thetraining dataset can be checked to ascertain its validity before it isused to ward off a potential training attack, or to help avoidinadvertent immunizations of one or more instances of malicious codefrom detection.

At 304, a set of distinct n-grams in the training dataset is identified.In some embodiments, the distinct n-grams are stored in a datastructure, such as a Bloom filter.

In order to address this collision issue, in some embodiments, a Bloomfilter can be configured to have multiple hash functions. By requiringall of the multiple hash functions to be set for verifying the presenceof each item in the Bloom filter, the false positive rate can bereduced.

At 306, an input dataset is received. As in 206, the input dataset canbe network data packets or frames, files containing data in a variety oftypes and formats, a stream of bytes or tokens of different lengths,and/or other suitable dataset. Also, as in 206, the input dataset can bea test dataset that is designed to measure how well a content anomalydetection model is trained, or a stream of incoming bytes that should bescrutinized before reaching its destinations.

At 308, the number of distinct n-grams identified in the input datasetthat are also contained in the training dataset are counted. In someembodiments, the distinct n-grams contained in the training dataset arestored in a Bloom filter. In some embodiments, for example, when adistinct n-gram contained in the input dataset is observed for the firsttime, the distinct n-gram is hashed to verify its presence in the Bloomfilter using one or more hash functions. If the hash function(s) verifythe presence of the distinct n-gram in the Bloom filter, then a counterassociated with the input dataset is incremented.

At 310, the total number of n-grams contained in the input dataset iscounted. In some embodiments, a counter is incremented each time ann-gram contained in the input dataset is observed regardless of whetherit has been observed already, or whether it has been previously observedduring the training phase. In some embodiments, 308 and 310 can beperformed at the same time.

At 312, an anomaly detection score of the input dataset is computedusing the number of distinct n-grams identified in the input datasetthat are also contained in the training dataset and the total number ofn-grams contained in the input dataset. In some embodiments, the anomalydetection score is computed using the following formula:

${{{Detection}\mspace{14mu}{Score}} = \frac{T_{d}}{T_{t}}},$where T_(d) is the number of distinct n-grams identified in the trainingdataset that are also contained in the input dataset and T_(t) is thetotal number of n-grams contained in the input dataset.

FIG. 4 is a simple illustration of a method 400 for generating andtraining a frequency distribution-based anomaly detection model inaccordance with some embodiments. At 402, a normal training dataset isreceived. As in 202, the normal training dataset can be network datapackets or frames, files containing data in a variety of types andformats, or a stream of bytes or tokens of different lengths. Also, asin 202, the normal training dataset can be checked to ascertain itsvalidity before it is used to ward off a potential training attack, orto help avoid inadvertent immunizations of one or more instances ofmalicious code from detections.

In some embodiments, the normal training dataset is randomly partitionedinto several parts in order to make crafting malicious code even moredifficult. In some embodiments, a random or secret combination of theseparts can be used to produce a distinct anomaly detection model. Byusing this random or secret combination of parts, attackers will beunable to know precisely which parts of the input dataset they shouldpad cause it to appear normal.

In some embodiments, partitions of the training datasets and inputdatasets can be chosen so that each partition has a minimum size. Forexample, in some embodiments, a random binary mask can be used togenerate random partitions, where each bit defines whether acorresponding partition is to be used (1) or not (0). In someembodiments, a string of contiguous 0's and 1's in the random binarymask is made to be at least m bits long (or another suitable size). Forinstance, any contiguous string in a random binary mask can be made tobe at least 10 bits. Additionally or alternatively, in some embodiments,the number of partitions that are used can be similar or equal in sizeto the number of partitions that are not used. Thus, in the binary maskexample above, the length of strings of contiguous 0's may be chosen toclosely match the length of strings of contiguous 1's.

At 404, one or more training datasets containing known instances ofmalicious code are received. In some embodiments, such instances ofmalicious code can be captured from prior attacks. Such prior attackscan be mimicry attacks, training attacks, and/or combinations thereof.As in 202, the training dataset(s) containing malicious code can benetwork data packets or frames, files containing data in a variety oftypes and formats, a stream of bytes or tokens of different lengths,and/or various other suitable datasets.

In some embodiments, as in 402, the training dataset(s) containingmalicious code are randomly partitioned into several parts that can beidentified more closely with one or more particular types of maliciouscode. In some embodiments, each of the training datasets includes one ormore particular types of malicious code.

At 406, a mixture of n-grams of different sizes is obtained. In someembodiments, a particular mixture of n-grams is kept secret to thwartmimicry attacks. If, for instance, an attacker does not know the mixtureof different sizes of n-grams being used by a content anomaly detector,the attacker would not know how to hide malicious code to avoiddetection.

In some embodiments, furthermore, one or more n-grams in the mixture arerandomly chosen from a pool of n-grams of different sizes and,therefore, the content of the secret mixture is known only to a contentanomaly detector that is used to train content anomaly detection modelsand detect anomalies using those models. For example, a mixture ofn-grams containing 2-grams, 3-grams, 7-grams, and 9-grams can berandomly chosen and used to serve as the core data for trainingdetection models.

At 408, an appearance frequency of each distinct n-gram of a mixture ofn-grams of randomly chosen sizes in the normal training dataset iscomputed. For example, for a mixture of randomly chosen n-gramsconsisting of 2-grams, 4-grams, and 5-grams, the appearance frequency ofeach distinct 2-gram, 4-gram, and 5-gram can be computed, respectively.In some embodiments, for example, the appearance frequency is computedvia the following formula:

${{F(i)} = {\log\left( \frac{x_{i}}{N_{t}} \right)}},$where x_(i) is the count of each unique n-gram i, N_(t) is the number oftypes of data (e.g., normal data, data containing malicious code, etc.),and F(i) is a measurement of an appearance frequency of each n-gram i.

At 410, the uniformity of distribution of each distinct n-gram in agroup of n-grams in the normal training dataset, or a group of itsparts, is computed. This is a measure of the entropy of the distinctn-gram. The uniformity of distribution can be used to find those uniquen-grams that frequently appear across the normal training dataset or agroup of its parts. In some embodiments, the uniformity of distributionis computed via the following formula:

${{U(i)} = {\frac{1}{\log\left( N_{i} \right)}{\sum\limits_{i}{P_{i}{\log\left( \frac{1}{P_{i}} \right)}}}}},$where P_(i) is a probability that n-gram i appears in the normaltraining dataset, N_(i) is an appearance frequency of the n-gram i, andU(i) is a term that measures the uniformity of distribution of then-gram i in the normal training dataset.

At 412, the uniformity of distribution of each distinct n-gram of amixture of n-grams in the training dataset(s) containing malicious codeis computed. The uniformity of distribution can measure how uniformlydistributed each unique n-gram is across the training dataset(s). Insome embodiments, uniformity of distribution can be computedproportionally to the inverse of the uniformity of distribution of theunique n-gram across the training dataset(s) and, therefore, thoseunique n-grams that appear more uniformly across the training dataset(s)are given less weight. In some embodiments, the term is computed via thefollowing formula:

${{A(i)} = {1 - \left( {\frac{1}{\log(L)}{\sum\limits_{i}{P_{ij}{\log\left( \frac{1}{P_{ij}} \right)}}}} \right)}},$where P_(ij) is a probability that a unique n-gram i appears in atraining dataset of type j, L is the number of different types oftraining dataset(s), and A(i) is a term that measures the uniformity ofdistribution of the n-gram i in one or more training datasets containingmalicious code.

At 414, a set of the most-heavily-weighted distinct n-grams of any sizeis determined. In some embodiments, the weight of each distinct n-gramin a group of n-grams in the training datasets is computed. For example,the weight can be computed via the following formula:W(i)=F(i)*U(i)*A ^(d)(i),where d is an exponent designed to adjust A(i) to produce diminishingreturns with scaling to large numbers of training datasets of differenttypes. In some embodiments, a predetermined number of distinct n-gramsof any size having the most-heavy weights is selected. For example, aset of M most-heavily weighted n-grams of any size can be selected. Insome embodiments, d can be 3 or 5.

FIG. 5 is a simple illustration of a method 500 for classifying an inputdataset using a content anomaly detection model that has been trainedusing method 400 in accordance with some embodiments. As shown, at 502,an input dataset is received. As in 206, the input dataset can benetwork data packets or frames, files containing data in a variety oftypes and formats, or a stream of bytes or tokens of different lengths.Also, as in 206, the input dataset can be a test dataset that isdesigned to measure how well a content anomaly detection model istrained, or a stream of incoming bytes that should be scrutinized beforereaching its destinations.

At 504, as in 408, an appearance frequency of each of distinct n-gramscontained in the input dataset is computed. In some embodiments, as in408, an appearance frequency of each of distinct n-grams in a group ofrandomly chosen n-grams in the input dataset is computed. At 506, as in410, the uniformity of distribution of each distinct n-gram in a groupof n-grams in the input dataset is computed.

At 508, as in 414, a set of the most-heavily weighted distinct n-gramsof any size is determined. In some embodiments, as in 414, the weight ofeach distinct n-grams in a group of n-grams in the input dataset iscomputed and a predetermined number of distinct n-grams having the mostweights is selected.

At 510, the input dataset is classified using the most-heavily weighteddistinct n-grams contained in the input dataset and the most-heavilyweighted distinct n-grams contained in the training dataset. Forexample, the most-heavily weighted distinct n-grams contained in thetraining dataset can be used to determine a threshold value. In someembodiments, the sum of the weights of the selected distinct n-grams iscomputed and tested against the threshold value. In some embodiments,the input dataset is classified as normal if the sum of the weights isgreater than the threshold value. In some embodiments, the thresholdvalue is adjustable.

FIG. 6 is a simple illustration of a method 600 for generating andtraining a binary-based probabilistic anomaly detection model inaccordance with some embodiments. As shown, at 602, a training datasetis received. As in 202, the training dataset can be network data packetsor frames, files containing data in a variety of types and formats, or astream of bytes or tokens of different lengths. Also, as in 202, thetraining dataset can be checked to ascertain its validity before it isused to ward off a potential training attack, or to help avoidinadvertent immunizations of one or more malicious code from detections.

In some embodiments, the training dataset is a set of themost-frequently occurring n-grams of any size that have been observedduring previous trainings by monitoring and recording a content flowwithin a network environment. For example, n-grams of size 1-10 can beused to determine the 1,000 most-frequently occurring n-grams of anysize from 1-10 to generate the training dataset. In some embodiments,the training dataset comprises only the most-frequently occurringn-grams that are extracted from data packets that are free of processingerrors. This can help avoid including packets that contain previouslyseen malicious code, to which servers are no longer vulnerable, in thetraining dataset.

At 604, the number of occurrences of distinct n-grams of different sizesin the training dataset is computed. For example, the number of previousoccurrences of 2-grams, 3-grams, and 5-grams in the training dataset canbe computed by counting every 2-gram, 3-gram, and 5-gram that iscontained in the training dataset, respectively.

At 606, a pseudo count associated with the number of occurrences of eachtype of n-gram is obtained to give some probability mass to eachpossible n-gram of each type that may be observed. In some embodiments,the pseudo count is predetermined and fixed. In some embodiments, it canbe adjusted by users. For example, a pseudo count associated with thenumber of occurrences of 2-grams, 3-grams, and 5-grams, respectively,may be obtained from a user. In some embodiments, the pseudo count maybe set equal to one.

At 608, a total number of each type of n-gram in the training dataset iscomputed. In some embodiments, for example, the total number of 2-grams,3-grams, and 5-grams in the training dataset is counted, respectively,using three separate counters.

At 610, the maximum number of distinct n-grams in the training datasetis computed. For 5-grams in bytes, for instance, the maximum number ofdistinct 5-grams is 256 to the fifth power, or 1024 billion, which canbe a rather large number with respect to memory and processor usages. Insome embodiments, therefore, the maximum number of distinct n-grams canbe adjusted. For example, 5-grams from a mixture of n-grams of size 1-10that are not chosen as the top most-frequently occurring grams arecounted. The count is then added to the number of 5-grams chosen as partof the top-most-frequently occurring grams to generate an adjustedmaximum number of distinct 5-grams in lieu of 1024 billion.

At 612, the total number of distinct n-grams of each type in thetraining dataset can be computed. For example, every distinct 2-gram,3-gram, and 5-gram in the training dataset can be counted, respectively,using three separate counters.

At 614, a smoothing factor that estimates the likelihood of observing apreviously unseen n-gram is computed. In some embodiments, the factor iscomputed via the following formula:

${C = \frac{N}{N + L - k_{0}}},$where C is a smoothing factor, N is the total number of a given type ofn-gram in the training dataset, L is the maximum number of distinctn-grams in the training dataset, and k₀ is the number of distinctn-grams in the training dataset.

In some embodiments, the smoothing factor is adjusted when L is toolarge. For example, the smoothing factor can be dominated by L if L istoo large. In some embodiments, therefore, the smoothing factor iscomputed via the following formula:

${C = \frac{N}{N + \beta}},$where β is a constant value. In some embodiments, β can be chosen byusers. In some embodiments, β is set to the value of k₀ when k₀ isapproximately equal to N.

At 616, the probability that each type of n-grams will be found in thetraining dataset is computed. In some embodiments, if n-gram of type ihas been previously observed, the probability is computed via thefollowing formula:

${{P(i)} = {\frac{\alpha + N_{i}}{{\kappa_{0}\alpha} + N}C}},$where α is a smoothing factor, N_(i) is the number of occurrences ofi-gram (i.e., n-gram of size i), κ₀ is the number of distinct n-grams inthe training dataset, N is the total number of a given type of n-gram inthe training dataset, C is a smoothing factor, and P(i) is a probabilitythat i-grams are fond in the training dataset.

In some embodiments, if an n-gram of type i has not been observed, theprobability is computed via the following formula:

${{P(i)} = {\frac{1}{L - \kappa_{0}}\left( {1 - C} \right)}},$where L is the maximum number of distinct n-grams in the trainingdataset.

At 618, a consistency score of n-grams of different sizes is computed,respectively. In some embodiments, the consistency score is computed viathe following formula:Consistency Score=Log(P _(i))+Log(L),where P_(i) is a probability that i-grams are fond in the trainingdataset and L is the maximum number of distinct i-grams in the trainingdataset.

In some embodiments, the order in which the different size grams areprocessed is taken into consideration for various optimizations. Forexample, processing the largest grams first may provide for storageoptimization. This may be the case because, if the set of the largestgrams is stored in a tree, for instance, a data structure can providethe means of representing information about the smaller grams withouthaving to generate a separate data structure for each of the differentsize grams.

In some embodiments, a content anomaly detector using a detection modelgenerated using method 600 is used to classify an input dataset. Given aset of consistency scores for different sizes of n-grams in a trainingdataset, for example, the content anomaly detector can extract differentsizes of n-grams from the input dataset, compute a consistency score foreach of the extracted n-grams, and classify the input data using thescores. In some embodiments, for example, the input dataset is regardedas abnormal if a sum of the consistency score of one or more extractedn-grams falls below a threshold value. In some embodiments, thethreshold value is determined using the set of consistency scores fordifferent sizes of n-grams in a training dataset.

FIG. 7 is a schematic diagram of a system 700 for outputting a datasetbased on anomaly detection in accordance with some embodiments. Asillustrated, system 700 includes a network 702, data traffic 704, adetector 706, and a data structure 708.

Network 702 can be a local area network (LAN), a wide area network(WAN), a wireless network, the Internet, and/or other suitable networksfrom which malicious attacks can be launched.

Data traffic 704 can include one or more network data packets, dataframes, one or more files that contain various types of data, such astext, graphic images, sound samples, video samples, andcomputer-executable codes, a stream of data in bytes or a stream ofother suitable symbols or tokens in one or more communication sessions,and/or other data in suitable formats.

Trainer and detector 706 can be various devices capable of performingsome or all of the functions described above in connection with FIG.1-6. For example, trainer and detector 706 can be implemented in acomputer, a digital processing device, a server, a processor, phone, apersonal data assistant, an email device, and/or various other suitabledevices.

Data structure 708 can be a data structure that allows for storing dataas described above in connection with the descriptions of FIG. 1-6. Datastructure 708 may be implemented in random access memory (RAM), flashmemory, a disk drive, optical media, and/or various other suitablestorage technologies.

Although the invention has been described and illustrated in theforegoing illustrative embodiments, it is understood that the presentdisclosure has been made only by way of example, and that numerouschanges in the details of implementation of the invention can be madewithout departing from the spirit and scope of the invention, which isonly limited by the claims which follow. Features of the disclosedembodiments can be combined and rearranged in various ways.

What is claimed is:
 1. A method for outputting a dataset based uponanomaly detection, the method comprising: receiving, using a hardwareprocessor, an input dataset; determining, using the hardware processor,an anomaly detection score by comparing n-grams that each represent asequence of n bytes in the input dataset with training n-grams, whereinthe anomaly detection score is indicative of the presence of anomalousn-grams in the input dataset; and causing, using the hardware processor,the input dataset to be outputted based on the anomaly detection score.2. The method of claim 1, wherein the anomaly detection score isdetermined by applying a content anomaly detection model to the receivedinput dataset.
 3. The method of claim 2, wherein the content anomalydetection model is a frequency distribution-based detection model thatdetermines a plurality of appearance frequencies, and wherein each ofthe plurality of appearance frequencies corresponds to one of thetraining n-grams.
 4. The method of claim 2, wherein the content anomalydetection model is a binary-based detection model that determines anumber of distinct n-grams in the input dataset and a total number ofn-grams contained in the input dataset.
 5. The method of claim 1,wherein the training n-grams include a first plurality of distincttraining n-grams and a second plurality of distinct training n-grams. 6.The method of claim 5, further comprising: determining a first pluralityof appearance frequencies corresponding to the first plurality ofdistinct training n-grams; and determining a second plurality ofappearance frequencies corresponding to the second plurality of distincttraining n-grams; and determining a first anomaly detection score basedon the first plurality of appearance frequencies and a second anomalydetection score based on the second plurality of appearance frequencies.7. The method of claim 6, further comprising: determining which of thefirst anomaly detection score and the second anomaly detection score ishigher; and outputting the input dataset based on the anomaly detectionscore that is higher.
 8. A system for outputting a dataset based uponanomaly detection, the system comprising: a hardware processor that:receives an input dataset; determines an anomaly detection score bycomparing n-grams that each represent a sequence of n bytes in the inputdataset with training n-grams, wherein the anomaly detection score isindicative of the presence of anomalous n-grams in the input dataset;and causes the input dataset to be outputted based on the anomalydetection score.
 9. The system of claim 8, wherein the anomaly detectionscore is determined by applying a content anomaly detection model to thereceived input dataset.
 10. The system of claim 9, wherein the contentanomaly detection model is a frequency distribution-based detectionmodel that determines a plurality of appearance frequencies, and whereineach of the plurality of appearance frequencies corresponds to one ofthe training n-grams.
 11. The system of claim 9, wherein the contentanomaly detection model is a binary-based detection model thatdetermines a number of distinct n-grams in the input dataset and a totalnumber of n-grams contained in the input dataset.
 12. The system ofclaim 8, wherein the training n-grams include a first plurality ofdistinct training n-grams and a second plurality of distinct trainingn-grams.
 13. The system of claim 12, wherein the hardware processor isfurther configured to: determine a first plurality of appearancefrequencies corresponding to the first plurality of distinct trainingn-grams; and determine a second plurality of appearance frequenciescorresponding to the second plurality of distinct training n-grams; anddetermining a first anomaly detection score based on the first pluralityof appearance frequencies and a second anomaly detection score based onthe second plurality of appearance frequencies.
 14. The method of claim13, wherein the hardware processor is further configured to: determinewhich of the first anomaly detection score and the second anomalydetection score is higher; and output the input dataset based on theanomaly detection score that is higher.
 15. A non-transitorycomputer-readable medium containing computer-executable instructionsthat, when executed by a processor, cause the processor to perform amethod for outputting a dataset based upon anomaly detection, the methodcomprising: receiving, using a hardware processor, an input dataset;determining, using the hardware processor, an anomaly detection score bycomparing n-grams that each represent a sequence of n bytes in the inputdataset with training n-grams, wherein the anomaly detection score isindicative of the presence of anomalous n-grams in the input dataset;and causing, using the hardware processor, the input dataset to beoutputted based on the anomaly detection score.
 16. The non-transitorycomputer-readable medium of claim 15, wherein the anomaly detectionscore is determined by applying a content anomaly detection model to thereceived input dataset.
 17. The non-transitory computer-readable mediumof claim 16, wherein the content anomaly detection model is a frequencydistribution-based detection model that determines a plurality ofappearance frequencies, and wherein each of the plurality of appearancefrequencies corresponds to one of the training n-grams.
 18. Thenon-transitory computer-readable medium of claim 16, wherein the contentanomaly detection model is a binary-based detection model thatdetermines a number of distinct n-grams in the input dataset and a totalnumber of n-grams contained in the input dataset.
 19. The non-transitorycomputer-readable medium of claim 15, wherein the training n-gramsinclude a first plurality of distinct training n-grams and a secondplurality of distinct training n-grams.
 20. The non-transitorycomputer-readable medium of claim 19, wherein the method furthercomprises: determining a first plurality of appearance frequenciescorresponding to the first plurality of distinct training n-grams; anddetermining a second plurality of appearance frequencies correspondingto the second plurality of distinct training n-grams; and determining afirst anomaly detection score based on the first plurality of appearancefrequencies and a second anomaly detection score based on the secondplurality of appearance frequencies.
 21. The non-transitorycomputer-readable medium of claim 20, wherein the method furthercomprises: determining which of the first anomaly detection score andthe second anomaly detection score is higher; and outputting the inputdataset based on the anomaly detection score that is higher.